1. Field of Invention
This invention relates to optical signal processing and, more particularly, to the use of fractional Fourier transform properties of lenses to correct unfocused effects of lenses in photographs, video, or other types of captured images. The system and method herein can be applied to conventional lens-based optical image processing systems as well as to systems with other types of elements obeying Fractional Fourier optical models and as well to widely ranging environments such as integrated optics, optical computing systems, particle beam systems, radiation accelerators, and astronomical observation methods.
2. Background of the Invention
A number of references are cited herein; these are provided in a numbered list at the end of the Detailed Description. These references are cited as needed through the text by reference number(s) enclosed in square brackets. Further, the cited disclosure contained within reference [1-18] is hereby incorporated by reference.
The Fourier transforming properties of simple lenses and related optical elements is well known and heavily used in a branch of engineering known as xe2x80x9cFourier Opticsxe2x80x9d [1,2]. Classical Fourier Optics [1,2,3,4] uses lenses or other means to take a first two-dimensional Fourier transform of an optical wavefront, thus creating at a particular spatial location a xe2x80x9cFourier planexe2x80x9d wherein the amplitude distribution of an original two-dimensional optical image becomes the two-dimensional Fourier transform of itself. This is less well known that the far simpler area of classical geometric Optics [1,3] where lenses and related objects are used to change the magnification of a two-dimensional image according to the geometric relationship of the classical xe2x80x9clens-law.xe2x80x9d In an earlier paper, it was shown that between the geometries required for Classical Fourier Optics and classical geometric optics the action of a lens or related object acts on the amplitude distribution of an images as the fractional power of the two-dimensional Fourier transform The fractional power of the fractional Fourier transform is determined by the xe2x80x9cfocal lengthxe2x80x9d characteristic of the lens or related object and the separation distances among the lens, the source image, and the observed image.
The Fractional Fourier transform has been independently discovered several times over the years [5,7,8,9,10] and is related to several other mathematical objects such as the Bargmann transform [8] and the Hermite semigroup [13]. As shown in [5], the most general form of optical properties of lenses and other related elements [1,2,3] can be transformed into a Fractional Fourier transform representation. This fact, to, has been apparently independently rediscovered some years later and worked on steadily ever since (see for example [6]), expanding the number of optical elements and situations covered. It is important to remark, however, that the lens modeling approach in the later long ongoing series of papers view the multiplicative-constant phase term in the true form of the Fractional Fourier transform [as a problem or annoyance and usually omit it from consideration; this is odd as, for example, it is relatively simply to take the expression for lenses from [2] and repeat the development in [5] based on the simplified expression in [1] and exactly account for this multiplicative-constant phase term.
The invention provides for corrections of the effects of misfocusing in recorded or real-time image data by means of fractional Fourier transform operations realized optically, computationally, or electronically. Herein the xe2x80x9ctermxe2x80x9d image refers to both still-images (such as photographs, video frames, video stills, movie frames, etc.) and moving images (such as motion video and movies).
The principal aspect of the invention uses the inherent Fractional Fourier transform properties of lenses or related elements or environments, such as compound lenses or graded-index materials or environments, to correct unfocused effects of lenses in photographs, video, or other types of captured images. Exploited is the xe2x80x9calgebraic unitary groupxe2x80x9d property of the fractional Fourier transform which allows for simple characterization of the exact inverse operation for the initial misfocus.
Another key aspect of the invention provides for relevant fractional Fourier transform operations to be accomplished directly or approximately by means of optical, numerical computer, digital signal processing, or other signal processing methods or environments.
Another aspect of the invention provides for small misfocus corrections to be made by approximating the fractional Fourier transform with its low-order derivatives with respect to its fractional power.
Another aspect of the invention provides for approximation methods which leverage Hermite function expansions which can advantageous in that the orthogonal Hermite functions diagonalize the Fourier transform and Fractional Fourier transform yielding the two-fold results:
throughout the entire optical system the amplitude and phase affairs of each Hermite function are completely independent of those of the other Hermite functions
the Hermite function expansion of a desired transfer function will naturally have coefficients that eventually tend to zero, meaning that to obtain an arbitrary degree of approximation in some situations only a manageable number of Hermite functions need be handled explicitly.
A further aspect of the invention allows the power of the fractional Fourier transform to be determined by automatic methods, and these automatic methods may include edge detection elements and provisions for overriding in part or entirety by a human operator.
A further aspect of the invention provides for the fraction Fourier transform power to be determined entirely by a human operator.
Another aspect of the invention provides for the fractional Fourier transform to be computed or approximated via chirps combined with conventional Fourier transforms using appropriately scaled variables.
Yet another aspect of the invention provides for at least one pre-computed power of the fractional Fourier transform to used in computing or approximating higher powers of the fractional Fourier transform.
Yet another aspect of the invention provides for combining numerical correction of video camera lens misfocus with video decompression algorithms so as to increase performance and reduce required misfocus-correction computation.
The system and method herein can be applied to conventional lens-based optical image processing systems as well as to systems with other types of elements obeying Fractional Fourier optical models and as well to widely ranging environments such as integrated optics, optical computing systems, particle beam systems, radiation accelerators, and astronomical observation methods.
In commercial products and services areas, the invention can be incorporated into film processing machines, desktop photo editing software, photo editing web sites, VCRs, camcorders, desktop video editing systems, video surveillance systems, and video conferencing systems, as well as in other types products and service facilities.
The incorporation of the method of the invention allows for the recovery of sharpened images from photographs, video, movies, or other types of captured images where the image was captured out-of-focus. Because a high quality lens or lens-system operates on the amplitude distribution of the source image as a two-dimensional fractional Fourier transform, the xe2x80x9calgebraic unitary groupxe2x80x9d property of the fractional Fourier transform allows for the exact calculation of the inverse operation for an initial lens or lens-system mis-focus. Additional mathematical properties of the fractional Fourier transform allow for several means of approximation meaningful in the economic embodiments of the invention. The system and method provided herein enable economic and wide-ranging implementation for after-capture correction of image misfocus.